Method and apparatus for performing privacy masking by reflecting characteristic information of objects

ABSTRACT

An image masking method is provided. The method includes: extracting an object from an image; obtaining characteristic information about the extracted object by analyzing the extracted object; determining whether the extracted object is a masking target according to an input setting value or the obtained characteristic information; and performing masking such that the obtained characteristic information is reflected on the extracted object, in response to determining that the extracted object is the masking target among a plurality of objects extracted from the input image, wherein the setting value is set by an input designating at least a partial region in the input image, and wherein in the determining whether the extracted object is the masking target, an object positioned in the at least a partial region is determined as the masking target among the extracted objects.

CROSS-REFERENCE TO THE RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.16/889,241, filed Jun. 1.2020, which is a continuation of U.S.application Ser. No. 16/239,775, filed Jan. 4, 2019, which claimspriority from Korean Patent Application No. 10-2018-0027792, filed onMar. 9, 2018, in the Korean Intellectual Property Office, the disclosureof which are incorporated herein by reference in their entireties.

BACKGROUND 1. Field

Apparatuses and methods consistent with exemplary embodiments of theinvention relate to image processing by extracting objects andcharacteristic information about the objects from an image andperforming privacy masking on the objects reflecting the characteristicinformation.

2. Description of the Related Art

Generally, a surveillance system has been widely used in various placesincluding banks, stores, and residential areas. Such a surveillancesystem may be used for crime prevention and security purposes, butrecently, has been also used to monitor pets or children in real time.The most popular surveillance system is a closed circuit television(CCTV) system photographing a desired region using a camera enabling auser to watch the desired region by monitoring an image photographed bythe camera.

The surveillance system basically performs surveillance for thatpurpose, but may perform object masking to cover or obscure an objectfor the purpose of privacy protection and the like. For example, when anautomated teller machine (ATM) is photographed by a camera at a bank,masking is performed so that an area in which a password is input iscovered, thereby preventing acquisition of secret information.

Korean Patent Registration No. 10-0588170 (hereinafter, referred to asPrior Art 1) is an invention relating to a method for setting objectmasking in a surveillance system. The Prior Art 1 includes a techniqueof displaying a masking block preset by a user overlapping an object.Prior Art 1 uses square-shaped blocks selected in a single color to maskthe object.

Korean Patent Registration No. 10-0877747 (hereinafter, referred to asPrior Art 2) is also an invention relating to object masking. Prior art2 performs masking using a filtering method for blackening maskingareas, a blurring method for mapping red, green and blue (RGB) values toaverage values of neighboring pixels, and a mosaic method formosaicking.

However, in the Prior arts 1 and 2, since the main purpose is privacyprotection, the masking is overly to prevent distinguishing a maskedobject or determining a motion of the object, and thus, an objectmonitoring function, which is the original purpose of a monitoringsystem, may be deteriorated.

SUMMARY

Exemplary embodiments of the inventive concept provide an image maskingmethod and apparatus having both a privacy protection function and anobject monitoring function.

The exemplary embodiments provide an image masking method and apparatusfor performing masking by reflecting characteristic information aboutobjects.

According to an exemplary embodiment, there is provided an image maskingmethod in which a processor and a memory are included and an operationis controlled by the processor. The image masking method may include:extracting an object from an image; obtaining characteristic informationabout the object; determining whether the object is a masking targetaccording to the characteristic information; and performing masking suchthat characteristic information about the object is reflected on theobject, in response to determining that the object is the maskingtarget.

According to an exemplary embodiment, there is provided image maskingmethod which may include: extracting an object from an image; obtaininginformation about the object comprising characteristic information aboutthe object; determining whether the object is a masking target accordingto a setting value and the information; and performing masking such thatthe characteristic information about the object is reflected on theobject, in response to determining that the information about the objectsatisfies the setting value.

According to an exemplary embodiment, there is provided monitoringdevice including an input/output interface configured to receive animage captured by a camera, and at least one processor. The at least oneprocessor may include: an object extractor configured to extract anobject from the image; an object analyzer configured to obtaininformation including characteristic information about the object; anobject determiner configured to determine whether the object is amasking target according to a setting value and the information; and anobject processor configured to perform masking in the image such thatthe characteristic information about the object is reflected on theobject, in response to determining that the information about the objectsatisfies the setting value.

The setting value may include at least one of color, shape, motion,identity, position, size, appearance time point, and appearancefrequency of the object in the image.

The object processor may generate a masking object by reflecting thecharacteristic information on the object, and perform the masking on theobject determined as the masking target using the generated maskingobject.

The masking object may be changed according to a change in thecharacteristic information about the object, and the masking object maybe an avatar or a silhouette on which the characteristic information isreflected.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other aspects and features of the inventive concept willbecome more apparent by describing in detail exemplary embodimentsthereof with reference to the attached drawings, in which:

FIG. 1 illustrates a surveillance system according to an embodiment.

FIG. 2 illustrates a monitoring device according to an embodiment.

FIG. 3 illustrates an embodiment for obtaining characteristicinformation about an object at an object analyzer.

FIGS. 4 and 5 illustrate an embodiment of determining a masking targetby an object determiner.

FIGS. 6 and 7 illustrate an embodiment of performing masking by anobject processor.

FIG. 8 illustrates an embodiment of performing masking at a monitoringdevice.

FIG. 9 illustrates an embodiment of performing masking at the monitoringdevice.

FIG. 10 illustrates an embodiment of performing masking at themonitoring device.

FIG. 11 illustrates an embodiment of performing masking at themonitoring device.

FIG. 12 is a flowchart of an image masking method according to anembodiment of the present invention.

DETAILED DESCRIPTION OF THE EXEMPLARY EMBODIMENTS

Embodiments of the inventive concept will be described in detail withreference to the accompanying drawings. It will be noted that all theembodiments described herein are exemplary. These embodiments will bedescribed in detail in order to allow those skilled in the art topractice the inventive concept. It should be appreciated that variousembodiments of the inventive concept are different, but are notnecessarily exclusive. For example, specific shapes, configurations, andcharacteristics described in one embodiment may be implemented inanother embodiment without departing from the spirit and the scope ofthe inventive concept. In addition, it should be understood thatpositions and arrangements of individual components in each disclosedembodiment may be changed without departing from the spirit and thescope of the inventive concept. Therefore, the detailed descriptionprovided below should not be construed as being restrictive. Inaddition, the scope of the inventive concept is defined only by theaccompanying claims and their equivalents if appropriate. Throughout thedrawings, the same reference numerals will refer to the same or likeparts.

Unless otherwise defined, all terms including technical and scientificterms used herein have the same meaning as commonly understood by one ofordinary skill in the art to which the inventive concept belongs. Itwill be further understood that terms, such as those defined in commonlyused dictionaries, should be interpreted as having a meaning that isconsistent with their meaning in the context of the relevant art and thepresent disclosure, and will not be interpreted in an idealized oroverly formal sense unless expressly so defined herein.

The terms used herein is for the purpose of describing particularembodiments only and is not intended to be limiting. As used herein, thesingular forms “a,” “an” and “the” are intended to include the pluralforms as well, unless the context clearly indicates otherwise. It willbe further understood that the terms “comprise”, “include”, “have”, etc.when used in the present disclosure, specify the presence of statedfeatures, integers, steps, operations, elements, components, and/orcombinations of them but do not preclude the presence or addition of oneor more other features, integers, steps, operations, elements,components, and/or combinations thereof.

Hereinafter, various embodiments of the inventive concept will bedescribed in detail with reference to the accompanying drawings.

FIG. 1 illustrates a surveillance system according to an embodiment. Thesurveillance system may include an image capturing device 3, an imagestorage device 2, and a monitoring device 1. An image captured by theimage capturing device 3 may be stored in the image storage device 2and/or displayed by the monitoring device 1.

The image capturing device 3 is a component for capturing an image, andvarious cameras may be used. Various types of cameras such as a zoomtype camera, a dome type camera, a pan/tilt/zoom (PTZ) camera, aninfrared (IR) camera, and a fisheye camera without being limited theretomay be used for the image capturing device 3.

The image capturing device 3 may be configured to be capable ofwired/wireless communication with the image storage apparatus 2 and themonitoring device 1 for transmission and reception of the captured imageand related data.

The image storage device 2 may be constituted by a network videorecorder (NVR), a digital video recorder (DVR), a central monitoringsystem (CMS), and a video management system (VMS), without being limitedthereto, as a device capable of storing the captured image.

The monitoring device 1 may be a CMS, a VMS, a personal computer (PC), amobile device, or the like, as a device capable of displaying thecaptured image.

Although the imaging capturing device 3, the image storage device 2, andthe monitoring device 1 are shown in FIG. 1 as separate devices, thedevices may be constituted as a single device. For example, the imagestorage device 2 and the monitoring device 1 may be constituted by oneVMS.

In the monitoring system according to the embodiment of the inventiveconcept, at least one of the imaging device 3, the image storage device2, and the monitoring device 1 may have an image masking function.Hereinafter, a method of performing a masking operation on themonitoring device 1 will be described.

FIG. 2 illustrates a monitoring device according to an embodiment. Themonitoring device 1 may include an input/output (I/O) interface 10, anobject extractor 40, an object analyzer 50, an object determiner 60, anobject processor 70, an image storage 20, a display 30, a database 80,and a user interface 90.

The I/O interface 10 may receive an image and transmit the receivedimage to the image storage 20, the display 30, or the object extractor40. The I/O interface 10 may receive the image from the image capturingdevice or the image storage device. Here, the I/O interface 10 mayinclude any one or any combination of a digital modem, a radio frequency(RF) modem, a WiFi chip, and related software and/or firmware.

The image received by the I/O interface 10 may be a pre-masked image ora non-masked image. The I/O interface 10 may transmit a received imageto the object extractor 40 if masking is required according to a type ofthe received image, or transmit the received image to the image storage20 or the display 30 if masking is not required or necessary.

For example, if the received image is already masked and additionalmasking is not required, the I/O interface 10 may transmit the receivedimage to the image storage 20 and the display 30 in order to store andoutput the received image. Otherwise, if the received image is alreadymasked, but additional masking is required, or if the received image isa non-masked image, the I/O interface 10 may transmit the received imageto the object extractor 40 for masking or additional masking of thereceived image.

The image storage 20 is a device for storing an image, and a storagemedium may be used for storing data. The storage medium which may beused for the image storage 20 may include a hard disk drive (HDD), asolid state drive (SSD), a compact flash (CF), and a secure digital (SD)card, but the kind thereof is not limited thereto.

The display 30 may be constituted by a liquid crystal display (LCD), anorganic liquid crystal display (OLED), a cathode ray tube (CRT), or aplasma display panel (PDP) as a device of outputting an image to ascreen, but the kind thereof is not limited thereto.

An image transmitted to the object extractor 40 through the I/Ointerface 10 may be masked through the object analyzer 50, the objectdeterminer 60, and the object processor 70. That is, the objectextractor 40, the object analyzer 50, the object determiner 60, and theobject processor 70 perform a series of operations for masking usinggeneral video analysis (VA), an image processing algorithm, and thelike.

The object extractor 40 may perform an operation of extracting an objectfrom an image. The object extracted by the object extractor 40 mayinclude all objects distinguished from a background in the image. Forexample, the object extracted by the object extractor 40 may be aperson, an animal, a vehicle, or the like.

The object extractor 40 may extract an object for each frame of theimage or extract an object for a frame having a large amount of imageinformation such as an I-frame.

The object analyzer 50 may perform an operation of analyzingcharacteristic information about the object extracted by the objectextractor 40. The characteristic information may include informationabout color, shape, motion, position, size, and/or identity of theobject, not being limited thereto.

The color information about the object means a color or a patternrepresenting the object in the image. The object analyzer 50 may extracta color or a pattern having the highest specific gravity constitutingthe object or extract an average value (for example, an RGB averagevalue) of the color constituting the object to acquire the colorinformation. In addition, the object analyzer 50 may extract a color ora pattern for each divided region by dividing the object or extract acolor or a pattern constituting a specific region of the object toacquire the color information.

In the color information, the color may be classified and extracted asRGB values or names of colors such as red and black, and the pattern maybe classified and extracted as names of patterns such as a dot, astripe, and the like.

For example, if the object is a person, the object analyzer 50 maydivide the person into a face, a hair, a top, and a bottom, and extracta color or a pattern for each part to acquire the color information.Alternatively, when the object is a vehicle and a specific regioncorresponds to a frame or body constituting an appearance of thevehicle, the object analyzer 50 may extract the color of the frame orbody of the vehicle to acquire the color information.

The shape information about the object means a shape of the objectprojected on the image. The object analyzer 50 may extract a boundaryline between the object and the background in the image to acquire theshape information.

The motion information about the object means information about aposture and an action taken by the object. The object analyzer 50 mayextract the position of a specific point, such as a face, a hand, and afoot, from the object to acquire information about the posture and theaction of the object.

The position information about the object means a point where the objectis positioned. The object analyzer 50 may acquire the positioninformation by calculating coordinates at which the object is positionedbased on an arbitrary point of the image. Alternatively, the objectanalyzer 50 may divide the image into a plurality of sections andacquire a section where the object is positioned as the positioninformation. Alternatively, the object analyzer 50 may acquire, as theposition information, the background where the object is positioned onthe image.

For example, the object analyzer 50 may acquire, as the positioninformation, coordinates at which the object is positioned with respectto the center of the image. Alternatively, the image may be divided intotwo sections based on the center, and a ‘left section’ may be acquiredas the position information when the object is positioned in the leftsection. Alternatively, when the image is obtained by capturing thekitchen and the living room indoor, and the object is positioned in thekitchen, the object analyzer 50 may acquire the ‘kitchen’ as theposition information.

The size information about the object means a size of the objectprojected on the image. The object analyzer 50 may calculate horizontaland vertical lengths of a square in contact with the outermost line ofthe object to acquire the lengths as the size information. Thehorizontal and vertical lengths of the square may be measured in lengthunits or the number of pixels.

The identity information about the object means information about a typeof the object such as a person, an animal, and a vehicle, and/orinformation about an identity of the object such as a name, a title, anda product name. If the object is a person, information about gender,age, face shape, or impression may also be included. In the case of ananimal or another object, identity information that can be separatelydiscriminated may be included. The object analyzer 50 may extractfeature points of the object and compare the feature points of theobject with the database 80 to acquire the identity information aboutthe object.

For example, the object analyzer 50 extracts the feature points of theperson's face in the image and compares the extracted feature pointswith the database 80 to acquire the name of the matched person.

The object determiner 60 may perform an operation of determining whetherthe object extracted by the object extractor 40 is an object to bemasked. The object determiner 60 may select all objects extracted by theobject extractor 40 as a masking target. Otherwise, the objectdeterminer 60 may select a specific object as the masking target fromthe extracted objects according to the characteristic informationacquired by the object analyzer 50 and a setting value input to themonitoring device 2.

The setting value may be a pre-input value or a value input by a userthrough the user interface 90. The setting value may be a value forcharacteristic information (color, shape, motion, position, size, and/oridentity). The setting value may be a value specifying a specific regionin the screen. Also, the setting value may be a value related to time,and an object appearing at a specific time duration, an object appearingat a specific time point, an object appearing at a specific number offrames, or the like may be selected as the masking target. The settingvalue may also indicate sensitivity for detecting a dynamic object, andthe sensitivity may be differently designated according to a region inthe image.

Here, the sensitivity is a reference value for detecting a motion in theimage. When a motion occurs in the image, a corresponding value isobtained, and the obtained value is compared with a predeterminedsensitivity value, and if the obtained value satisfies the predeterminedsensitivity value, it is determined that the motion occurs in the image.

For example, when the user adjusts up the sensitivity, a light motion isdetected, and when the sensitivity is adjusted down, only a heavy motionis detected.

The object determiner 60 may select an object having the characteristicinformation satisfying the setting value as a masking target.

For example, when the setting value is ‘top: red’, the object determiner60 may select an object, in which the color information about the top ofthe extracted object is classified in red, as a masking target.

As another example, when the setting value is ‘position: kitchen’, theobject determiner 60 may select an object, in which the positioninformation about the extracted object is classified as the kitchen, asa masking target.

As yet another example, when the setting value is ‘name: R. Green’, theobject determiner 60 may select a person, in which the name is R. Greenof the extracted object, as a masking target.

As still another example, when the setting value is a value designatinga specific region in the screen, the object determiner 60 may select anobject positioned in the specific region as a masking target.

As still yet another example, when the setting value is a valuedesignating the sensitivity, the object determiner 60 may select adynamic object satisfying a predetermined sensitivity as a maskingtarget.

As still yet another example, when the setting value is a valuedesignating sensitivity in a specific region in the screen, the objectdeterminer 60 may select a dynamic object satisfying the designatedsensitivity among the objects positioned in the specific region as amasking target.

That is, the user may set the sensitivity to a region where privacy isto be protected to be high, and the object having even a slight motionmay be designated as a masking target. In this way, in a place such as ahospital divided into a private space where privacy is to be protectedand a public space where the privacy is not protected, the sensitivitymay be set to be low in the public space to enhance a monitoringfunction, and set to be high in the private space to enhance a privacyprotection function.

As still yet another example, when the setting value is a valuedesignating a time point, the object determiner 60 may select an objectappearing at the designated time point as a masking target.

The object processor 70 may perform masking on an object determined asthe masking target. The object processor 70 may mask an object selectedbased on characteristic information about at least one of color, shape,motion, position, size information, and identity of the object.Different from an actual object, a masking object is an object generatedby reflecting the characteristic information about the actual object.The masking object reflects the characteristic information about theobject, and when the characteristic information about the object ischanged, the masking object may be changed according to the change incharacteristic information about the object.

The masking object may be an avatar or a silhouette in which thecharacteristic information about the object is reflected.

The object processor 70 may perform masking on an object by covering theobject with an avatar or a silhouette, which is a masking object createdon the object, or masking by processing the object to be transparent andinserting an avatar or a silhouette, which is a masking object generatedat a position of the object.

For example, if the color information about an object is ‘face: black,hair: black, top: red, bottom: black’, masking may be performed using anavatar constituted by black face and hair, a red top, and a blackbottom. At this time, when the color information about the object ischanged, such as when the object takes off clothes or changes theclothes, the color information about the avatar as the masking objectmay be changed. In addition, when the identity information about theobject is ‘gender: female and age: middle age’, the masking may beperformed using an avatar representing a middle-aged woman. Furthermore,it is possible to adjust the size of the avatar according to the sizeinformation about the object, and perform the masking by adjusting theposition of the avatar according to the position information about theobject.

In another example, the masking may be performing by generating the samesilhouette as the object according to the shape information about theobject. The interior of the silhouette may be filled with a random coloror a color matched with the color information about the object.

As such, since the object is masked using an avatar or a silhouetteobtained by reflecting the characteristic information about the object,it is possible to protect the privacy of the object while transmittinginformation about the object to the user (an object monitoringfunction).

An image masked by the object processor 70 may be transmitted and storedto the image storage 20 or may be transmitted and displayed at thedisplay 30.

As such, since masking on an object is performed using an avatar or asilhouette reflecting characteristic information about the object at theobject processor 70, the user easily distinguishes the masked object,and information about impression, motion, and identity of the objectsmay be transmitted to the user.

Since the object extractor 40, the object analyzer 50, the objectdeterminer 60 and the object processor 70 perform a series of operationsfor masking every frame (or every frame of a specific period), an avataror a silhouette which is changed according to a motion of an object maybe generated and masked.

Embodiments for processing images at the object analyzer 50, the objectdeterminer 60, and the object processor 70 will be described in detailwith reference to the above description.

FIG. 3 illustrates an embodiment for acquiring characteristicinformation about an object in the object analyzer. In FIG. 3 , theobject extractor 40 extracts a first object 200 and a second object 300from a received image 100. The object analyzer 50 extractscharacteristic information about the first object 200 and the secondobject 300 extracted by the object extractor 40.

First, the object analyzer 50 extracts feature points of the face of thefirst object 200 and compares the extracted feature points with thedatabase 80 to analyze whether the first object 200 is a registeredperson. In the embodiment, the first object 200 is a person who is notregistered in the database 80, and the object analyzer 50 analyzes theidentity of the first object 200 as ‘person’.

The object analyzer 50 divides the first object 200 into a face, a hair,a top, and a bottom, and acquires color information by extracting acolor that occupies the largest portion in each region. In theembodiment, the object analyzer 50 acquires color information of ‘face:yellow, hair: black, top: black, bottom: black’.

The object analyzer 50 may acquire the size information through thehorizontal and vertical lengths of a square 201 in contact with theoutermost line of the first object 200. In the embodiment, the objectanalyzer 50 acquires size information of ‘horizontal length: 220 mm,vertical length: 340 mm’.

The object analyzer 50 may acquire a region where the first object 200is positioned as position information. In the embodiment, the backgroundof the image 100 is divided into a public space 110 corresponding to awaiting room and a private space 120 corresponding to a care room in thehospital. The object analyzer 50 may acquire the ‘public space’ as theposition information according to the background in which the firstobject 200 is positioned or acquire coordinates ‘(300, 600)’ where acentral point of the first object 200 is positioned based on a lowerleft end of the image as the position information.

Next, the object analyzer 50 extracts feature points of the face of thesecond object 300 and compares the extracted feature points with thedatabase 80 to analyze whether the second object 300 is a registeredperson. In the embodiment, the second object 300 is a person who isregistered in the database 80, and the object analyzer 50 analyzes ‘R.Green’, the name of the second object 300 as the identity information.

The object analyzer 50 divides the second object 300 into a face, ahair, a top, and a bottom, and acquires color information by extractingan average value of the colors in each region. In the embodiment, theobject analyzer 50 acquires color information of ‘face: yellow, hair:black, top: red/dot, bottom: black’.

The object analyzer 50 may acquire size information through thehorizontal and vertical lengths of a square 301 in contact with theoutermost line of the second object 300. In the embodiment, the objectanalyzer 50 acquires size information of ‘horizontal length: 130 mm,vertical length: 300 mm’.

The object analyzer 50 may acquire a region where the second object 300is positioned as position information. In the embodiment, the objectanalyzer 50 may acquire the ‘private space’ as the position informationaccording to the background in which the second object 300 is positionedor acquire coordinates ‘(1200, 500)’ where a central point of the secondobject 300 is positioned based on a lower left end of the image as theposition information.

FIGS. 4 and 5 illustrate an embodiment of determining a masking targetby the object determiner.

First, referring to FIG. 4 , a user may designate the first object 200as a masking target through the user interface 90 while monitoring theimage 100. The object determiner 60 may select the first object 200 as amasking target according to an input in which the user selects the firstobject 200 and designates the first object 200 as the masking target.Accordingly, the object processor 70 performs masking on the firstobject 200 among the first object 200 and the second object 300.

Next, referring to FIG. 5 , the user may designate the specific region130 as a masking region through the user interface 90 while monitoringthe image 100. The object determiner 60 may select the second object 300positioned in the designated region 130 as the masking target accordingto an input of the user designating the region 130. If the first object200 enters the designated region 130 and the second object 300 deviatesfrom the designated region 130, the object determiner 60 selects thefirst object 200 as the masking target.

In this way, since only an object positioned in a designated region inan image can be masked, selective masking based on a space other thanthe object is possible. In a place such as a hospital divided into aprivate space where privacy is to be protected and a public space wherethe privacy is not protected, masking is not applied to the public spaceto enhance a monitoring function, and the masking is applied to theprivate space to enhance a privacy protection function.

The object determiner 60 may determine only an object havingcharacteristic information satisfying a setting value among the sametype of objects as a masking target. Only a person among a person, ananimal, and a vehicle may be determined as a masking target, and only anobject having characteristic information satisfying a setting valueamong a person, an animal, and a vehicle may be determined as themasking target. Alternatively, only one object may be determined as themasking target or only a predetermined number of objects may bedetermined as the masking target.

FIGS. 6 and 7 illustrate an embodiment of performing masking by theobject processor.

First, referring to FIG. 6 , the object processor 70 may perform maskingon an object using an avatar that is obtained by reflectingcharacteristic information acquired by the object analyzer 50. Theobject processor 70 may generate an avatar 210 in which colorinformation of ‘face: yellow, hair: black, top: black, bottom: black’ ofthe first object 200 is reflected. In addition, the avatar 210 changedaccording to a change in the motion of the first object 200 may begenerated by reflecting motion information about the first object 200.When the color information is changed by taking off or changing clothes,the color of clothes of the object may also be changed. As such, theavatar may also be changed according to the change in the characteristicinformation about the object.

Similarly, an avatar 310 in which color information ‘face: yellow, hair:black, top: red/dot, bottom: black’ of the second object 300 isreflected may be generated. Alternatively, since the second object 300is a registered person, an avatar having a pre-selected shape may begenerated to match the corresponding person.

Next, referring to FIG. 7 , the object processor 70 may generate asilhouette 220 according to shape information about the first object200. The interior of the silhouette 220 may be filled with black, whichis a color of the highest specific gravity of the first object 200. Inaddition, the silhouette 220 changed according to a change in thecontour of the first object 200 may be generated by reflecting the shapeinformation about the first object 200.

Similarly, the object processor 70 may generate a silhouette 320according to shape information about the second object 300. The interiorof the silhouette 320 is divided into a face region, a hair region, atop region, and a bottom region according to color information about thesecond object 300, and each region may be processed as yellow, black,red/dot, and black. Alternatively, since the second object 300 is aregistered person, the interior of the silhouette 320 may be filled witha pre-selected color or pattern to match the corresponding person.

As such, since an avatar or a silhouette is generated by reflectinginformation about at least one of color, shape, motion, identify, andthe like of an object, the a user may determine an attire of the objecta posture of the object, and/or whether the object is a registeredperson, through the avatar or the silhouette. Therefore, the user mayeasily distinguish the object even though the object is masked, andconfirm the detailed motion of the object.

The embodiments in which masking is performed in the monitoring device 1will be described according to the above descriptions. In the followingembodiments, it is assumed that the first object 200 and the secondobject 300 have the characteristic information illustrated in FIG. 3 .

FIG. 8 illustrates a first embodiment of performing masking at themonitoring device. In the first embodiment, the object determiner 60determines all objects in the image 100 as a masking target.

When the image 100 is received through the I/O interface 10, the objectextractor 40 extracts the first object 200 and the second object 300from the background.

The object analyzer 50 analyzes and acquires information about at leastone of color, shape, motion, position, size, and identity with respectto the extracted objects 200 and 300.

As the masking target is preset to all objects, the object determiner 60selects all extracted objects 200 and 300 as the masking target.

The object processor 70 generates the avatars 210 and 310 obtained byreflecting the color information and the size information about theobjects 200 and 300, and performs masking using the avatars 210 and 310generated according to the position information about the objects 200and 300.

A masked image 100′ may be output through the display 30 or may bestored in the image storage 20.

FIG. 9 illustrates an embodiment of performing masking at the monitoringdevice. In this embodiment, the object determiner 60 determines acorresponding object as a masking target according to a setting valuewhich is previously input.

The user presets a value for characteristic information about at leastone of color, shape, motion, position, size, and identity, through theuser interface 90. It is assumed that the user sets ‘name: R. Green’ asthe identity information.

The object determiner 60 selects a masking target by comparing thecharacteristic information acquired by the object analyzer 50 with thesetting value. The first object 200 and the second object 300 areobjects having the characteristic information illustrated in FIG. 3 ,and the object determination unit 60 determines that the second object300 as the masking target.

The object processor 70 generates an avatar 310 obtained by reflectingthe characteristic information about the second object 300 determined asthe masking target. The avatar 310 may be an avatar obtained byreflecting the color information about the second object 300 or anavatar pre-selected to match with ‘R. Green’.

FIG. 10 illustrates an embodiment of performing masking at themonitoring device. In this embodiment, the object determiner 60determines a masking target according to object designation by a user.

The user may monitor the image 100 and designate a masking targetthrough the user interface 90. It is assumed that the user designatesthe first object 200 as a masking target.

The object determiner 60 selects the first object 200 designated by theuser as the masking target, and the object processor 70 performs maskingby generating the avatar 210 obtained by reflecting characteristicinformation about the first object 200 selected as the masking target.

FIG. 11 illustrates an embodiment of performing masking at themonitoring device. In this embodiment, the object determiner 60determines a masking target according to region designation of the user.

The user may monitor the image 100 and designate a region to be maskedthrough the user interface 90. It is assumed that the user designates aregion positioned on the right side of the screen as the masking region.

The object determiner 60 selects the second object 300 positioned in theregion designated by the user as the masking target, and the objectprocessor 70 performs masking by generating the avatar 310 obtained byreflecting characteristic information about the second object 300selected as the masking target.

Also, the user may designate a masking region, and designate sensitivityto the masking region to be higher than other regions.

The object determiner 60 selects the second object 300 positioned in theregion designated by the user and satisfying the designated sensitivityas the masking target, and the object processor 70 performs masking bygenerating the avatar 310 obtained by reflecting the characteristicinformation about the second object 300 determined as the maskingtarget.

FIG. 12 is a flowchart illustrating an image masking method according toan embodiment. Detailed description of the image masking methodaccording to this embodiment corresponds to the detailed descriptionsabout the monitoring device 1 of FIGS. 1 to 11 .

In operation 1210, an object is extracted from an input image. Inoperation 1220, characteristic information about the extracted object isobtained by analyzing the extracted object. In operation 1230, it isdetermined whether the extracted object is a masking target according toat least one of a setting value and the characteristic information. Inoperation 1240, masking is performed such that the characteristicinformation about the object is reflected to the object determined asthe masking target among objects in the image.

The setting value is set by an input designating at least a partialregion in the image, and in the determining whether the extracted objectis the masking target, an object positioned in the designated region maybe determined as the masking target among the objects in the image.

Alternatively, in the determining whether the extracted object is themasking target, only an object having characteristic informationsatisfying the setting value among the same type of objects may bedetermined as a masking target.

Furthermore, in the performing the masking, the masking is performedwith respect to the object determined as the masking target using amasking object such as an avatar and/or a silhouette reflecting thecharacteristic information, and the masking object may be changedaccording to a change in characteristic information about the object.

The embodiments of the inventive concept can be embodied as computerreadable codes on a computer readable recoding medium. The computerreadable recoding medium includes all types of recording devices inwhich data readable by a computer system are stored.

Examples of the computer readable recoding medium include a read-onlymemory (ROM), a random access memory (RAM), a compact disc (CD)-ROM, amagnetic tape, a floppy disk, an optical data storage device and thelike, and further include a medium embodied by a carrier wave (forexample, transmission via the Internet). Further, the computer readablerecording medium is distributed in a computer system connected through acomputer network and computer readable codes may be stored and executedin the distributed manner. Further, functional programs, codes, and codesegments for embodying the inventive concept may be easily deduced byprogrammers in the technical field to which the inventive conceptbelongs.

At least one of the components, elements, modules or units (collectively“components” in this paragraph) represented by a block in the drawingssuch as FIG. 2 may be embodied as various numbers of hardware, softwareand/or firmware structures that execute respective functions describedabove, according to an exemplary embodiment. For example, at least oneof these components may use a direct circuit structure, such as amemory, a processor, a logic circuit, a look-up table, etc. that mayexecute the respective functions through controls of one or moremicroprocessors or other control apparatuses. Also, at least one ofthese components may be specifically embodied by a module, a program, ora part of code, which contains one or more executable instructions forperforming specified logic functions, and executed by one or moremicroprocessors or other control apparatuses. Further, at least one ofthese components may include or may be implemented by a processor suchas a central processing unit (CPU) that performs the respectivefunctions, a microprocessor, or the like. Two or more of thesecomponents may be combined into one single component which performs alloperations or functions of the combined two or more components. Also, atleast part of functions of at least one of these components may beperformed by another of these components. Further, although a bus is notillustrated in the above block diagrams, communication between thecomponents may be performed through the bus. Functional aspects of theabove exemplary embodiments may be implemented in algorithms thatexecute on one or more processors. Furthermore, the componentsrepresented by a block or processing steps may employ any number ofrelated art techniques for electronics configuration, signal processingand/or control, data processing and the like.

The embodiments of the inventive concept have at least the followingeffects.

Since objects are masked using an avatar or a silhouette obtained byreflecting characteristic information about the objects, it is possibleto protect the privacy of the objects while transmitting informationabout the object to a user (an object monitoring function).

The effects according to the inventive concept are not limited by thecontents exemplified above, and other various effects are included inthe inventive concept.

Those skilled in the art to which the inventive concept belongs will beable to understand that the inventive concept can be implemented inother detailed forms without changing the technical spirit or anessential characteristic. Therefore, it should be appreciated that theaforementioned exemplary embodiments described above are allillustrative in all aspects and are not restricted. The scope of theinventive concept is represented by claims to be described below ratherthan the detailed description, and it is to be interpreted that themeaning and scope of the claims and all the changes or modified formsderived from the equivalents thereof come within the scope of theinventive concept.

What is claimed is:
 1. An image masking method of a surveillance systemin which a processor and a memory are included and an operation iscontrolled by the processor, the image masking method comprising:extracting an object from an image; obtaining characteristic informationof the extracted object by analyzing the extracted object in an image;designating a specific region in the image based on a user input;comparing the characteristic information with a database to analyzewhether the extracted object is a registered person; selecting theextracted object as a masking target when the extracted object is notthe registered person; detecting whether the extracted object ispositioned in the specific region; and performing selective masking onthe extracted object according to whether the extracted object ispositioned in the specific region, wherein the performing of theselective masking includes: not performing masking on the extractedobject that is positioned outside of the specific region, and performingthe masking on the extracted object when the extracted object enters thespecific region.
 2. The image masking method of claim 1, wherein theobtained characteristic information is a characteristic information formonitoring the extracted object.
 3. The image masking method of claim 2,wherein in the performing masking on the extracted object, the obtainedcharacteristic information is reflected on the extracted object.
 4. Theimage masking method of claim 3, wherein a silhouette is displayed inthe image according to shape information about the extracted object. 5.The image masking method of claim 1, the characteristic information ofthe extracted object is displayed together with and adjacent to theimage.
 6. The image masking method of claim 3, wherein the obtainedcharacteristic information comprises information about at least one ofcolor, shape, motion, identity and position.
 7. The image masking methodof claim 1, wherein the performing the masking comprises: generating amasking object by reflecting the obtained characteristic information;and performing the masking on the extracted object determined as amasking target using the masking object.
 8. The image masking method ofclaim 7, wherein the masking object is an avatar on which the obtainedcharacteristic information is reflected.
 9. The image masking method ofclaim 8, wherein at least one of a face, a hair, a top, a bottom, agender, and an age of the avatar is determined according to the obtainedcharacteristic information.